COMMENT ⊗   VALID 00002 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002	Comments on:
C00018 ENDMK
C⊗;
Comments on:
		THE PSI TRACE EXPERT, by J. V. Phillips

**************************************************
GENERAL comments
**************************************************

There are some new, interesting ideas contained in this paper, plus a
few  ideas  which (although  not  novel) have  never  been explicitly
stated in print before. This paper should definitely be  accepted for
the 2nd Int'l.  Software Engineering Conference.

Although the  paper is acceptable in  its present form, I  advise the
author to  spend more time editing any future papers. If possible, he
should try to rewrite this paper, although I don't  recommend that as
a condition of acceptance.   The major defect in the paper is that of
very poor editing. By that, I mean:

	(i) frequent  use of  undefined terms  and jargon [e.g.,  INT
form].  Most of  this stems from the obvious fact that TU is just one
module in a larger system. This paper should be edited so that it can
"stand on its own".
	(ii) frequent  occurences of very awkward  English sentences.
Although  I, as reviewer, had  to take the time to  read a sentence 3
times if  necessary, the average  reader may  simply move on  without
comprehending  it.   [e.g., the  following two  senteces  follow each
other: "The scene fits.  Output fits." I parsed  the second one  just
like the  first, i.e.,  to mean  "the output  does in  fact fit  some
criteria", when in fact it was supposed to mean "print out the output
which deals with how x fit y"]
	(iii) occasional lapses into  naivete', ignorance of  earlier
sources,  and exaggerated  claims.  [e.g., "The  most  common way  of
transferring knowledge  between humans, be it in the form of concepts
or procedures,, is by using examples"].
	(iv) Failure to make clear  what the current state of  TU is.
The  reader is  never  quite sure  whether  this paper  reports on  a
running program, or merely an idea for a program.
	(v) Poor  placement of  sections.  [e.g., Section  3.3.1  and
3.3.2 are excellent sections, buried in the middle of the paper.]
	(vi) Poor abstract  (very hard to understand what  this paper
is all about),  and poor conclusions section (the material in Section
5 is  more  a  "summary"  of  the design).  What  (if  any)  ARE  the
conclusions?

**************************************************
SPECIFIC
**************************************************

Abstract: very hard to comprehend. Is this obfuscation synergetic?

p1: "The most common..." is pretentious and proabably inaccurate.

p2: Does TU  presume that the program  can naturally be written  as a
procedural  one  (linear  sequences of  actions,  loops,  and tests)?
Could TU  really infer,  say, MYCIN  -- a  production-system-oriented
program? Of course  the LISP compiler does produce  such a procedural
form,  but it  would be  very awkward  to infer  from traces.  If the
inferrer were  told to  try to infer  a production  system, his  task
would be greatly simplified.  Have you thought about what assumptions
TU is making re its  target program? How does  a small bit of  global
information --  like `it's  a production  system' help constrain  the
inference  process?  It's perfectly  OK that TU will  write a certain
kind of program only, but you should state that explicitly.

p2: You claim that "A single trace  is not enough" to infer TF. Is  a
trace just one  cycle of the outermost loop? Surely  if a long enough
session  were presented, a  HUMAN could infer the  program. Why isn't
this enough for TU?

p3: This sounds like TU can infer ANY program from its trace. Are you
really claiming  that? How about  the interesting and  useful program
URAND?  When claims of universality are involved, the conclusions are
necessarily trivial from  a pragmatic standpoint (e.g., see  the work
by Blum&Blum).

p3: Similar uses  of the phrase "natural language descriptions" imply
that TU  (or  at  least PSI)  can  speak English  and  other  tongues
fluently. At best, I imagine that TU  (PSI) claims to be usable by an
untrained, unprepared human, who will be led to conduct a dialogue in
a limited subset of English.

p3: Why is Natural Language  capitalized in only some places?  Do you
really mean a particular fixed subset of ENglish, when you capitalize
N.L.?

p4: Section 1.2.2  is either  trivial or  unfathomable, depending  on
one's  familiarity  witht  he  rest  of  the   PSI  system.  (To  me,
unfathomable).

p5:  What exactly is an  "INT form"? What is  a "form-changing rule"?
Give example of these things when you state them, if not sooner.

p6: This makes TU sound like a small, nonessential module in a larger
system.   If TU is this  unimportant, why isn't it  just described in
the  overall PSI  paper?   Probably,  you should  just skip  all this
material.

p8: Why do  you quote from  an as-yet-unpublished reference for  this
dialogue excerpt and for the TF program? The same excerpt (and target
program) are  given in  Lenat's  "Beings" paper.  See the  4th  IJCAI
proceedings, pp 130-1.

p8: "The scene fits the  concept so output fits to the  user" is hard
to  parse. Can PSI  really handle  these sophisticated constructions?
Even if  it  can,  is  this done  by  TU?  If not,  it  is  merely  a
distraction in this  paper. Keep the Natural Language  easy to parse,
please.

p11:  "concept" has hiterto been used  in a specific technical sense.
Now it  seems  to be  used in  a new,  colloquial  sense. The  first,
technical use  (part of TF) was never  stated explicitly. Considering
the emotional charge of that word, this is not a good policy. Even if
YOU call both  of these things concepts,  make up a separate  name to
keep the reader from confusing them.

p12-13: What observation?!  This is crucial! Was it by introspection,
by interviews  with experts,  by guessing,  by careful  psychological
studies, etc.?   There may be some new ideas  buried here, and you're
doing them  a disservice.  "Concept" here is not related to "Concept"
as a data structure in TF, is it?  By the way, what does TF stand for
(True/false? Theory-former?...)?

p14: What  if more  than 1 template  is suggested as  relevant? e.g.,
what about this sentence: "The initial values are input"?

p14: What  are  "slot-filling rules"?  Give  examples. How  many  are
there?   Do they  exist yet, or  are they  merely planned? Who  wrote
them, and how?  How can they be checked, extended, etc.? How general,
powerful, useful are they?  THis "fan-dancer" style of yours is quite
frustrating.

p14: Section 3.1.3: Does "value" mean "the name, the identity of"?

p15: What  is a collection?  plex? correspondence? This  is obviously
PSI jargon.   If you must use it, at least explain it. I infer that a
colllction is  a  1-level list,  a plex  is a  property  list, and  a
correspondence is a list of ordered pairs.  Is this even close?

p.17: Section  3.3: What if the target  program uses "processes", and
the only orderings on the events are PARTIAL orderings? WIll spurious
control branches be inferred?

p.17: Does `isomorphic to' really just mean "sufficient to serve as"?

p.17: Surely you don't mean  to say that TU is designed  to infer THE
control structure of  the particular target TF program(s) you've been
looking at?  Ther  might be many "natural" ways  to code up the  same
programin LISP, i.e., to write a program having a simialr trace but a
quite  different  control  structure.    If  1  TF programis  somehow
"distinguished", that is worth  explaining carefully.  It's OK  if TU
has  a specific ability,  as long as  that is  made clear.   In other
words, could TU  turn out several  very different control  structures
from the same  trace, or is there  really just one family  of control
structures that TU "assumes" the target program will fall into?

Section 3.3.1: an excellent section.

Section  3.3.2:  an  excellent  section.    Here  you  might  mention
explicitly that TU would not work well for Operationg systems, or for
production-based  programs   (MYCIN),  or  for  programs   using  the
agenda-like  mechanism for control (a la  Winograd and Bobrow's KRL),
where jobs get stored, and get executed inorder of priority.

p.20:  Section 3.3.3:  a  collection  of unintelligible  PSI  jargon,
again.  Explain this, give examples, or OMIT.

p23: I disagree that the  2nd is more likely. The 1st is more likely,
since it's shorter. If A and A-B are equally plausible, A is simpler,
hence (by Occam)  preferable to A-B. Of  course you know that  A-B is
"right"  for getting TF,  and this  has been pre-programmed  into TU.
Make these  hacks  clearer. Why  doesn't  TU consider  the  symmetric
difference between A and B, in fact? That works, in this case, and is
just as  valid as the set-difference between A and B. [above, A and B
represent "scene" and "concept model"].

Section 3.6: does  this (albeit short)  section add anything to  this
paper?   It rates perhaps 1  sentence, in an appendix  called "how TU
fits into PSI".

p25: What in the world is an "AAU Actor"? Does Carl Hewitt know? Does
such an actor belong to some union?

Section 4: After  reading this far in  the paper, why can't  I follow
section 4 in detail? Perhaps some comments are needed in the margins.

Section 5: These ar not conclusions. Are there any so far? It's OK if
not.

Appendix A: As long as you're  using up all these lines, why not  add
little comments  along the  right margin, to  explain what  the slots
mean?